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Abstract 

We produce a new basis for the Schur and Weyl modules associated 
to a row-convex shape D. The basis is indexed by new class of "straight" 
tableaux which we introduce by weakening the usual requirements for 
standard tableaux. Spanning is proved via a new straightening algorithm 
for expanding elements of the representation into this basis. For skew 
shapes, this algorithm specializes to the classical straightening law. The 
new straight basis is used to produce bases for flagged Schur and Weyl 
modules, to provide Groebner and SAGBI bases for the homogeneous co- 
ordinate rings of some configuration varieties and to produce a flagged 
branching rule for row-convex representations. Systematic use of super- 
symmetric letterplace techniques enables the representation theoretic re- 
sults to be applied to representations of the general linear Lie superalgebra 
as well as to the general linear group. 



1 Introduction 



Akin, Buchsbaum, and Weyman in [ ABW82] give a construction that associates 
a GLn-representation to any generalized shape like 



or p^ B or Cj-J"' or 



(1) 



Significant progress has been made by Reiner and Shimozono and by Laksh- 
mibai and Magyar in describing bases for these GL„-representations and for 
the associated flagged representations of the Borel subgroup of upper triangular 
matrices in GL„. As one expects, these bases are indexed by some subset of the 
generalized tableaux found by filling each cell in the generalized shape with a 
number from 1 to n. 
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The present paper shows how to construct a well-behaved straight basis for 
the representations associated to any row-convex shape, such as jj-P "', with 
no gaps in any row. In particular, we give a local condition for testing whether 
a tableau is straight, we give a straightening law that modifies only two rows 
at a time, and the basis we present reduces immediately to a flagged basis. 
The straight basis is distinct from the bases produced by Reiner-Shimozono 
and Lakshmibai-Magyar, but see [ 199 for some combinatorial and algebraic 
relationships between these bases. The straight bases provide a canonical choice 
of basis for certain row-convex and column-convex "almost-skew shapes." These 
shapes were shown by Woodcock in | W94 | to possess a class of easily flagged 
bases, but no method was presented for distinguishing a basis in this class or for 
straightening elements of the representation into a linear combination of basis 
elements. 

Results on flagged tableaux are deduced in Section ^ from the main theorem 
on straight bases. As shown in Section ^ the straight basis and straightening 
algorithm may be applied to produce quadratic Groebner bases and SAGBI bases 
for the homogeneous coordinate rings of certain configuration varieties. Further 
applications to commutative algebra may be found in [T97a| and |T9£]. Appli- 
cations to the representation theory of GLn, Bn, Sn, and the general linear Lie 
superalgebras are derived in Section |^ where a branching rule is produced for de- 
composing a row-convex GLn-representation in terms of GLn_i-representations. 

This paper studies the Schur and Weyl modules as special cases of the super 
Schur modules which we construct as submodules of the letterplace superalge- 
bra. All results in this paper are characteristic-free and the requisite background 
on superalgebras is detailed in Section ^ Much of the presentation in Section |^ 
is new and, we hope, accessible to the non-specialist. The construction proper is 
given in Section |^. Straight tableaux are introduced and independence is proved 
in Section ^. Section ||, the heart of the paper, presents the straightening algo- 
rithm. 



2 Polynomial superalgebras 

This section introduces the definitions required to make the main results of this 
paper characteristic free and applicable to Weyl modules. The reader concerned 
only with Schur modules in characteristic may safely take C and V to be the 
positive integers, N, (or finite subsets of N.) The set C may be thought of as 
the indexing the rows of a generic matrix (xij) and V indexes the columns. We 
may then take take Super {[C {V]) to be the polynomial ring whose variables 
are matrix entries Xij. The letterplace {i\j) is taken to be shorthand for Xij 
and the expression [ii, . . . ,ik\ji, ■ ■ ■ ,jk] is taken to be the determinant of the 
k X k minor {xi^_j^) of the matrix (xij). 

The constructions used in this paper take place inside polynomial superalge- 
bras over the integers, Z, that is inside tensor products of symmetric, exterior, 
and divided powers algebras. We construct the polynomial superalgebras over Z 
as Z-subalgebras of a symmetric algebra over the rationals, Q, tensored with an 
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exterior algebra over Q. Write the symmetric and exterior Z-algebras associated 
to a set C as Sym{C) and A(£). These are Z-subalgebras of the symmetric and 
exterior Q-algebras Sym(^{C) and Aq(£) associated to C. The divided powers 

algebra, 'Div{C) of a set C is the Z-subalgebra of SymQ{C) generated by all 
for all X £ £. 

We define a signed set to be a set C together with a a function | | : £ ^ Z2 . 
We say that elements in the preimage of are positively signed; we call this 
preimage £+. Elements in the preimage, , of 1 are said to be negatively 
signed. A signed set C endowed with a total order, <, is called a (signed) 
alphabet. For notational convenience, we define two new inequalities, <+ and <- 
on C. We say that a <+ b (respectively a <- b) when a < 6 or when a = b and 
\a\ = |6| = (respectively a = b and \a\ = \b\ = 1.) 

A superalgebra is simply an algebra with a Z2-grading. We construct a Q- 
superalgebra with the elements of a signed set as generators and such that the 
grading on these generators is | |. For any signed set £, define SuperQ{C) to be 
SymQ{C^) Aq{£^). Likewise, we define Super{C) to be VivlC^) ® A{£^); 
as above, we may consider this to be a Z-subalgebra of SuperQ{C). Given 
another signed set V, we will define the "letter-place" algebra, Super {[C {V]), 
to be a Z-subalgebra of SuperQdxa.dyaec.dev) where \xa.d\ = |a| + \d\. In 

particular, Super{[C \ V]) is the subalgebra generated by all Xa,d and by all ^f- 
with a,d € , i gN. This algebra is naturally isomorphic to 

A(£" xr+\+jC+ xV^)^ Sym{£- xV^)® Viv{£+ x V+) . 



We extend | | to a Z2 grading of Super {[C {V]). Following |GRS87|, we write 
the elements Xa,d of Super{[C \ V]) as the signed variables (ajd), and we will 
define the biproduct, {wi, . . . ,Wk\vi, . . . ,vu), of a pair of sequences w_ and y_ in 
C and V respectively. 

Definition 2.1 Given sequences w — wi, . . . ,Wk € C and v — vi, . . . ,Vk ^ V, 
define (w\v) = (wi, . . .,Wk\vi, ...,Vk)^ E^eSfe ("^^7(1) l^i) • ' ' {w„(^k)\vk) 

where n„ — il^{{i,j) : i < j, <T^^{i) > a^^{j), Wi,Wj are negative} 

+H={{i,i) ■ i > .j and Wcr{i), Vj are negative}. 

The following definition/proposition indicates that the biproduct can be thought 
of as a bilinear map on Super{C) x Super{V). 

Definition 2.2 Given sequences w = vui, . . . ,Wk & C and v — vi, . . . ,Vk & V, 
define {wi ■ W2 ■ ■ ■ Wk\vi ■ V2 ■ ■ ■ Vk) ^ {wi,W2, • . • , Wk\vi,V2, • . • ,Vk). Extend this 
by bilinearity to a map ( | ) : Super{L) x Super{P) Super{[C \ V]). 

It is straightforward to to check that this map is well-defined. 

In order to better handle divided powers as elements of a Z-subalgebra con- 
tained in a Q-algebra, we make the following definition. If w is a sequence in A 
for some signed set then we define c {w) \ to be riie^+ (#times i appears in w)!. 
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Call a monomial Af = ni('«b«) & SuperQ{[C | P]) sorted when pi < p2 < ■■■ 
and Pi = pi+i G implies U <+ k+i and dually pi — Pi+i G V~ implies 
li -> U+i- Super{[C I V]) is a free Z-module with basis consisting of the divided 
powers monomials {-^jjjyM^ for all sorted monomials M G Super q^{\C \'P]). 
Here c {M)\ — c ((Zi|pi), (/2IP2), ■ ■ ■)■■ We will consider two monomials (respec- 
tively divided powers monomials) the same when they differ by a nonzero scalar 
multiple (respectively a multiple of ±1.) 

Define a function Tab(w|w) when w_ (respectively v) is a fc-tuple of letters in 
C (respectively V) by 

Tab(«;i, ...,wuW,...,Wk)= . ,| , (_l)# {('-J):'>J^ ^-^^T'^^w\v). 

c [wjic [vy. 

Observe that the divided powers monomials occur with coefficient ±1 in the 
expansion of Tab(w|w) and that if wi <+ W2 <+ ■ ■ ■ <+ Wk and vi <+ W2 <+ • ■ ■ <+ 
Vk, then the basis element .-p-j- — yw riil'^^iK'*) appears with coefffcient 1. 



3 Schur modules, Weyl modules, generalizations 

In this section, we define our primary object of study, the super-Schur module as 
a Z-submodule of a letterplace algebra. The unsigned cases produce the Schur 



and Weyl modules of Akin-Buchsbaum-Weyman | ABW82 when C = Cr and 
C, = respectively. 

We define two tableaux associated to a shape. The first is useful for referring 
to cells in the shape and the second plays a fundamental role in our construction 
of the super-Schur modules. 

Definition 3.1 Let D be a shape. 

Dehne F{D) to be the tableau of shape D whose cells are labeled 1,2,3,... 
starting with the northmost cell in the leftmost column and continuing down 
the column, then down the second leftmost column, etc. In this paper, the signs 
of the letters in F{D) are irrelevant. 

A tableau of shape D is termed Deruyts if it is obtained by filling each cell 
in the diagram with the cell's column index viewed as a negative variable. We 
denote such a tableau by Der~{D). 

Shapes appearing in this paper are assumed, unless otherwise noted, to have 
first coordinate 1 in their top rows and second coordinate 1 in their leftmost 
columns. 



Example 3.1 



F 



36 _ 3-4- 

3-4- 



2 



3- 



1- 
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Definition 3.2 Suppose S and T are tableaux of the same shape. Let the word 
Si betheithrowofS andletTi betheithrowofT. DeGne[S\T] = JJ. Tab{Si\Ti). 
Hence [S\T] ^ 

Now suppose that T is a tableau of shape D. Suppose that C contains the set 
of letters present in T and that V~ contains the indices for all columns present 
in D. Define an element [T] G Super {[C \ V]) indexed by T by, 



m = 



T 



Der-{D) 



Example 3.2 Let £ = £- = {a,b,c,d,e, f,g} and let V = = {1,2,3,4}. 
a d 



Let T 



. Then 



a d 


34 ' 


h c e 


1 34 


/ 


3 


. 9 


1 



[T] 



In other words, [T] is a scalar multiple of (a(i|23)(6ce|123)(/|2)(g|l). The scalar 
in this example being 1, we have 

/ fdi3) m) \ I ^''^^ ^"'^^ ^''^^ 

[^]=^°M „3 „4 |(/I2)(5|l). 
V W^) W^) J \ (5|3) (5|4) 



Definition 3.3 Suppose that D is a shape. Define the super-Schur module 

S^{C) = spanz|[r] : shape(T) ^ V and T is filled with letters from £|. 

In the case that C is negative (respectively positive) then S^{C) is called the 
Schur (respectively Weyl) module associated with the diagram D. These terms 
are justified by the following result. A proof may be found in |T97a| . 



Proposition 3.4 Let R be a commutative ring. Let F be a free R-module of 
rank n. Let a be the Q/l-matrix having 1 's precisely where D has cells. 

If C = C~ has cardinality n, then R®zS^ {C~) = La{F), where La{F) 
is the Akin-Buchsbaum-Weyman "Schur functor" associated to the generalized 
shape matrix a. 

If C = C+ has cardinality n, then R(g)zS^{C+) = Ka{F), where Ka{F) is 
the Akin-Buchsbaum-Weyman "coSchur functor." □ 



Example 3.3 The Weyl module of shape '-0-' on positive letters a, b is spanned 

by 



aaa 




aab 




bbb ' 




abb 




abb 




aab 


b 




b 


7 


a 


5 


a 


5 


b 




a 



r aaa 1 
I a J 



bbb 
b 
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in the exterior algebra generated by the anti-commuting variables (a|l), (a|2), 
(a|3), {b\2), (613). The last two of the above skew-polynomials are identi- 

cally 0. In the next section we single out the first three elements as a basis. 



4 Row-convex diagrams and straight tableaux 

The usual bases for skew Weyl modules consist of the semistandard Young 
tableaux, namely all tableaux which weakly increase in their rows and strictly 



increase in their columns. Example 3.3 showed that this is not the case for 



more general shapes. Nevertheless, the basis of | ABW82 | for shape D skew 
Weyl modules indexed by standard Young tableaux of shape D has a number 
of properties we wish to preserve. In particular: 

1. The rows of the tableaux in the indexing set weakly increase. 

2. Knowing the number of times a letter appears in each column of a tableau 
in the indexing set determines that tableau. 

3. It is combinatorially "obvious" when a tableau is in the indexing set. 

4. The elements [T] where T is in the index set form a basis for module. 

5. There is an easy to describe algorithm for rewriting [T] in terms of basis 
elements. 



Property ^ underlies the SAGBi-basis algorithms of |3tu93|; in [W94| Woodcock 



shows that there must exist bases satisfying this property when D is "almost- 
skew." 

Only slightly more complicated shapes, F|-P for instance, fail to simultane- 
ously possess properties ^ ||, and ^. To see this, examine the Specht module 
associated to this shape. Recall that this is the subspace of the associated 
Schur module spanned by all tableaux containing letters 1~, 2~, 3~, 4~ with no 
repeats; here the indexing shape is transposed from the indexing shape used 



in | Sa91 |. This Specht module is isomorphic to the one indexed by rFP hence 
has dimension 5. However, there are only 4 tableaux of shape R-P satisfying 
conditions |l] and ||. 

We define a class of "straight" tableaux satisfying the above properties. The 
elements [T] where T is straight and of shape D will form a basis for the super- 
Schur module for any "row-convex" shape D. 

Definition 4.1 A row-convex shape, such as jj-P "', is a shape with no gaps 

in any row. I.E., if cells {r,i) and {r,k) are in a shape D, then (r, j) is in D, 
for all i < j < k. Since the constructions of section ^ are not sensitive to the 
order of rows in a diagram, we assume that the rows of a row convex diagram 
are sorted so that higher rows end at least as far to the right as lower rows. 
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We can denote any row-convex shape by A/m where A is a partition and m is 
a composition satisfying < Ai for aU i; a cell is in position (i, j) of A/m iff 
mi<j< K- 



Following [GRS87], and employing the notation for inequalities introduced 
on page a tableau T with entries in a signed set is standard when it (<+)- 
increases across rows and (<-)-increases down columns. 

I introduce the notion of a straight tableau of row-convex shape by slightly 
relaxing the usual conditions for standardness of a tableau. 

Definition 4.2 A row-convex tableau is called straight when 

1 . The contents of any row <+-increase from left to right, and 

2. Given two cells in the same column, say {i, k) and (j, k) for i < j, the 
entry in the top cell, {i, k), may be i+>)-larger than the entry in {j, k) (i.e. 
the cells form an inversion) only if cell {i,k — 1) exists and its content is 
{->)-larger than the content of {j, k). 

This definition amounts to requiring that the columns are as close as possible to 
(<-)-increasing, subject to the condition that the rows remain (<+)-increasing. A 
more precise version of the preceding fact is implicit in the correctness Algorithm 
Straight-Filling in Figure |l|. A tableau satisfying condition ^ is called row- 
standard and an inversion violating condition |^ is called a flippable inversion. 

Proposition 4.3 A skew tableau, T, is straight iff it is standard. 

Proof. Since a standard tableau has no inversions, it suffices to prove the only- 
if part. We prove the contrapositive. We can assume that T is row-standard. 
Suppose that the cells (i, fc), [j, k) with i < j are an inversion. Let fco be the least 
(leftmost) column such that (z, fcg), (j, fcg) is an inversion. If (i, feg — 1) exists then 
by skewness so does [j, kQ — \) and thus by assumption Ti k^^i <- Tj ^^^i <+ Tj^^j^ 
hence T is not straight. □ 



Corollary 4.4 The straight tableaux of skew shape with only positively signed 
letters are the usual semistandard Young tableaux. □ 



Definition 4.5 Given a tableau T its column word, ct is the word formed by 
reading the entries of T from bottom to top and left to right. Its modified 
column word is the word wt formed by writing the entries of the first column 
in weakly decreasing order followed by the entries of the second column in 
decreasing order, etc. 

We shall also occasionally require a reverse column word of T formed by 
writing the entries of the first column of T in increasing order then those of the 
second column in increasing order etc. 



A straighteng algorithm for row-convex tableaux. 


8 


Input: A word w' of length n, and an n-celled row-convex shape 


D. 


Output: A straight tableau T with w'j, — w' or "impossible" if no such tableau 


exists. 




Let Cj be the column index of the j in F{D). 




Let T be an empty tableau of shape D 




for k = 1 . . .n 




Let i be the smallest (northmost) index such that {i,Ck] 


e D is still 


empty and either there is no cell in position (i, — 1) 


or r,,cfc-i <+ w',,. 


if there is no such i then return "impossible" 




else T,^c, ^ Wfe- 




Figure 1: Algorithm Straight-Filling 





Theorem 4.6 IfT and T' are straight tableaux of the same shape, then T ^ T' 
implies wt 7^ wx' ■ More strongly, if there exists a straight tableau T of shape 
D with Wt — w then the algorithm Straight-Filling in Figure^ produces it. 

Proof. A tableau, T, produced by this algorithm must be straight. If in a fixed 
column, k, the letter y is inserted into row i by the algorithm while x <± y was 
inserted into row j > i, then it must be that Ti^k-i > x else cell Ti^k would 
have been available to x hence x would have been placed there. 

Now suppose that the algorithm produces a tableau T with reverse column 
word w' . Let c be as in the algorithm. Any tableau with reverse column word 
w' can be produced by a similar filling process. Define i so that reading through 
w' and inserting w'^ into cell (ik,Ck) gives the desired tableau. Let us assume 
that if w'f, appears in multiple cells in column ct that the first w'f, in w' is used 
to fill the northmost appearance in the column, the second is used to fill the 
second northmost appearance, etc. 

Let i be the filling sequence corresponding to T, this is the sequence pro- 
duced by the Algorithm Straight-Filling. Let i' be the filling sequence corre- 
sponding to some other tableau T'. Let ko be the smallest integer such that 
ika ^ i'f^g- So in filling T\ we have placed into cell {i'i^^^,Cko) when accord- 
ing to Algorithm Straight-Filling, it could have been put into (i/c(,,Cfco) where 
iko < i'kg- I3y necessity, in filling T', something (>)-larger than Wkg must be 
placed in {ikg,Cko). By our assumptions about repeated letters in the defini- 
tion of i, this inequality is strict. But these facts guarantee that the inver- 
sion {{iko: Cfco), {i'kg J Cfco)} of T' violates condition ^ in the definition of straight 
tableaux. 

The above argument says that if we try to create a straight tableau T with 
w'rp = w' by reading across w' and sequentially filling its letters into a tableau 
then at each step the choice of where to insert the letters is forced on us. If 
at any point during execution of the algorithm there is no place to put a letter 
which preserves row-standardness, then it is in fact not possible to find a straight 
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tableau with the designated column content and shape. This is precisely the 
circumstance under which "impossible" is returned. 

We conclude that not only does Straight-Filling produce a straight tableau, 
but any other tableau, T' having the same modified (equivalently reverse) col- 
umn word is not straight. □ 



Corollary 4.7 The matrix expressing the super-polynomials [T] indexed by 
straight tableaux as Z-linear combinations of divided powers monomials in the 
polynomial superalgebra is in echelon form with ±1 at each pivot. Hence the 
straight basis elements are linearly independent. 

We defer the proof in order to develop the appropriate orders on basis ele- 
ments and monomials. Monomials are ordered according to a generalization of 
the "diagonal term order" in | Stu93| which requires that the smallest monomial 
in det(^), where ^ is a minor of (xij), be the product of the elements on the 



diagonal. For compatibility with lexicographic order in Lemma [4.9| this is back- 
wards from the convention in commutative algebra which has Hil-^i,*) be the 
largest monomial in det(A). 

Definition 4.8 A diagonal term order on Super{[C \ P]) is 

1. A total order, -<, on monomials in Super{[C \ V]) such that for monomials 
m,m' ,n,n' , the relations m ~< m' and n ~< n' imply that mn ~< m'n' or 
mn = or m'n' = 0. 

2. The smallest monomial in a nonzero biproduct {ii . . . , . . . ,jk) with 
ii < ■ ■■ <ik and ji < ■ ■ ■ < jk is HiiiilJi)- 

The default diagonal term order, ^diag that we utilize is characterized below. 
We order letterplaces by {i\j) > {i'\j') when j < j' or when j = j' and 
i > i' . Let M,N € Super{[C \ V]) be two nonzero monomials. Suppose 
is the largest letterplace appearing to a different power in M and N. Write 
N~<diasM when M is divisible by a higher power of than is N. 

Example 4.1 Suppose C = {1^,2^} and V = {a+,x~}, then (2|a) > (l|a) > 
(2|x) > (l|a;). Further, we have 

(l|x) < < (2|a)(l|a;)2(2|a;) < (2|a)(2|a;)2(l|a;) < (2|a)(l|a)(l|2;)2. 

The following lemma is immediate. 

Lemma 4.9 A normalized monomial nf=i(*HjO i^ Super{[C \ V]) is a 
monomial written so that > in the default diagonal term order. 

For two normalized monomials, M = nf=i(*'bO ^^'^ ^ = n(=i(*ibO differing 
only in their letters, M < N in the default diagonal term order iff ii, . . . ,ik is 
lexicographically less than i'^, . . . ,i'j,. □ 



A straighteng algorithm for row-convex tableaux. 



10 



Definition 4.10 Let be the function taking a normalized monomial Yii^iiUlji) G 
Super{[C I P]) to ii, . . . ,ik- 



Definition 4.11 Given p e Super {[C \ P]) and an order -< on monomials, de- 
fine the initial monomial init^(p) ofp to be the smallest (divided powers) mono- 
mial appearing in p. 

Sometimes the phrase "initial term" will be used when the coefficient of the 
initial monomial is to be included. 

The following result says that in most cases the modified column word of T can 
be read directly from the smallest monomial appearing in [T]. 

Proposition 4.12 IfT is a tableau whose rows (<+ )-increase and whose columns 
contain no repeated positive letters, then 

Proof. Suppose [T] = J]; Tab(u;i,c,,i , Wi^c;, 2; ••■ i |ci,i, Ci, 2, Ci,iJ. The 
initial term (with coefRcient) of the ith multiplicand is ^^(''^j.Ci j l^ij)- and since 
positive letters never repeat in a column the product of these initial terms is 
nonzero and hence equals init^j.^^([T]). D 



Note that the initial term Y[i Y[ji'^i,ci j appearing above is (up to sign) 
a basis element in the monomial Z-basis for Super {[C \ P]) and we have proved 
the following. 

Proposition 4.13 If T is straight of shape D, and if ci is the index of the 
column of F{D) containing I then 

Y[i^iwir)i\ci) =imU,J[T]) 
I 

The coefRcient of the initial monomial of \T] is ±1. □ 



Corollary 4.14 Suppose T is a straight tableau, then wt = ^(init^^ ([T])). 
□ 

We now complete the proof of the independence result. 



Proof. {oi Corollary [O. ) Since Theorem 4.6 says that distinct straight tableaux 



have distinct modified column words, we conclude from Corollary 4.14 that 
if monomials are ordered by -<diag and the polynomials [T] corresponding to 
straight tableaux are ordered lexicographically by their modified column words, 
then the matrix expressing the [T] in terms of divided powers monomials is in 
echelon form with ±l's as pivots. □ 
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Corollary 4.15 Suppose p = J^i ctiTi is a linear combination of row-standard 
tableaux such that p = J2j Pj [^3] wiiere the Sj are distinct straight tableaux and 
where all tableaux have the same row-convex shape D. The smallest modified 
column word of a tableaux in the Sj 's is weakly larger (lexicographically) than 
the smallest modified column word appearing in the T/. 

Proof. Let c/ be the column of F{D) containing I. Suppose that WTi „ < 
for all i and suppose ws^^ < for all j ^ jQ-recall by Theorem [l.6| that 
distinct straight tableaux have distinct modified column words. We want to 
show WTig < ''^Sjg ■ Now because straight tableaux have distinct modified column 
words ^[/(''^•Sjo J*^') smallest monomial occurring in p. That means that 

it must appear in ct jli if that expression is expanded out to a polynomial 
in Super{[C \ V]). But if is always larger than wsj^ then no monomial as 
small as Yliiws^^ilci) can appear in J2i o^Ti- □ 

The next section shows that any [Ti] can be rewritten in the above 

fashion. 



5 A straightening algorithm 

We produce an explicit two-rowed straightening law for reducing any tableau to 
a linear combination of straight tableau. This algorithm, straighten-tableau 
shown in Figure ^ starts with a tableau T and returns a formal linear com- 
bination CiiSi of straight tableau with integer coefficients such that [T] = 
X^i Q^i['S'i]. In each step, the algorithm looks for a pair of rows containing a flip- 
pable inversion. If these exist, it applies the sub-algorithm row-straighten in 
Figure ^ to "straighten" these two rows via the Grosshans-Rota-Stein syzygies 



of Definition 5.1 



We provide an example of the straightening law below. 

Example 5.1 Let C = £^ = {1,2,..., 8}. In each step we shall look for a 
non-straight tableaux T and locate two rows (say ri above r2) in T containing 
a flippable inversion. In this example we will mark by a * every cell in row ri 
weakly right of the left most flippable inversion in those rows and every cell in 
row r2 that is weakly left of this flippable inversion and weakly right of a cutoff 
column ci . The cutoff ci indexes the leftmost column of row r2 such that either 
Tri.ci-i does not exist or r^j.^-i <+ Tr^^a- In this example, ci happens to 
always index the leftmost column in row r2. We mark the remaining elements 
in row ri by "'s. 

The Grosshans-Rota-Stein syzygies (proved for the commutative case C = 
L m [ pRS76t ) says that anti-symmetrizing all the *'d elements in T, is the 



same (up to sign) as collecting all the *'d elements into the row ri, replacing 
those removed from row r2 with these "'d elements, and anti-symmetrizing the 
• 'd elements. We shall repeatedly apply this identity. 
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Input: A row-convex tableau T. 

Output: J2t oct,S^ such that [T] = Oi^[S^\ where each is a straight tableau 
and at € Z. 

if T is straight then output T. 

else there exists a flippable inversion in some rows i,j 
Let Sk/^k • be the output of row-straighten^ 

Let N^, be 

(# pos. letters in Wk_ + # pos. letters in Tj) ■ (# pos letters in Ti^_i • • • Tj_i). 



Output ^^(— Ij^^/^K • straighten-tableau 



• --Tt-l ■ ■ ■ 
• • • • • • 

• • • Ti+i • • • 



V 



Figure 2: Algorithm straighten-tableau. Tj, Tj, etc. are the ith, jth, etc. 
rows of the tableau T. 



So, observing that the entries 4 and 2 form a flippable inversion, we first 
have, 



4*5* 
13 5 7 
2* 
3 8 



2*5* 
13 5 7 
4* 
3 8 



2*4* 
13 5 7 
5* 
3 8 



But the cells in column 3 and rows 2 and 3 of the first tableau on the right hand 
side now contain a flippable inversion. We straighten as follows, 



2 5 
1'3'5*7* 

4* 
3 8 



2 5 
1»3'4*7* 

5* 
3 8 



2 5 
1"3'4*5* 

7* 
3 8 



+ 



2 5 
1"4*5*7* 
3* 

3 8 



2 5 
3*4*5*7* 
!• 

3 8 



Now the first two tableaux above are straight, but the last two are not. We 
straighten the next to last tableau by, 



2 5 
1*4*5*7* 

3 

3*8 



2 5 
1*3*5*7* 



4*8 



2 5 
1*3*4*7* 

3 

5*8 



+ 



2 5 
1*3*4*5* 



7*8 



+ 



2 5 
3*4*5*7* 
3 

1*8 



and the last tableau by 



2*5* 
3 4 5 7 
1* 
3 8 



1*5* 
3 4 5 7 
2* 
3 8 



1*2* 
3 4 5 7 
5* 
3 8 



so 
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Input: A two-rowed row-convex tableau T = - which is row- 

standard but not straight. 

Output: J2k. ' such that 

Claim 1: [T] = "« I^'^l ^^i^^re a„ G Z and 

Claim 2: the column word of ...^^ ... is lexicographically larger than the column 
word of T. 

Let C2 be the index of the column containing the leftmost flippable inversion. 

Let ci be the smallest column such that ci > TO2 and either Wci-i <+ Wci 

or ci — 1 < mi (i.e. Wci^i does not exist.) 

Let C3 be the rightmost column such that = ■ 

if ci < C2 then 

Let YliL^i Pi.'^i' ~ Syzc2^c2+i,---,\i;ci,ci+i,---,c3{T)- 
Expansion ^ 0. 
for t G / 

if wt^ > wt then Expansion ^ Expansion + fS^T^ . 

else Expansion <— Expansion -\- (3^ ■ row-straighten(rt). 
Output Expansion. 
elseo Comment: ci — C2. 

Let Co be the leftmost column such that Vc^ +> ■ 
> Comment: cq < C2 ^ Wcq = Wc2- 

Let Y.,^iPi-T, = SyZco,co+ir-M-cuci+i.-.c3(T). 
Expansion <— 0. 
for t G / 

if Wt, > Wt then Expansion <— Expansion + P^T^. 

else Expansion ^ Expansion + /3t • row-straighten(rt). 
Output Expansion. 



Figure 3: Algorithm row-straighten. If £ = £~, then we will always 
have Wt, > so the algorithm will never recurse and instead could have 
directly output the expressions Syz{T). The expression Syz{T) is defined in 
Definition 5.1 
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The first step in verifying Algoritlim straighten-tableau is to prove that 
when T is replaced with J2i Pi^i by Algorithm row-straighten we have [T] = 
J2i Pi [Ti] ■ The second step involves showing that each Ti is somehow closer to 
being straight than was T. The first of these facts is an immediate consequence 
of the correctness of Algorithm row-straighten. This will come down to veri- 
fying the identities used in the preceding example. The second will follow from 
the correctness of row-straighten and the fact, proved in Proposition ^.4| , that 
given a tableau T and another tableau T' differing only in two rows then 
the column word of the two-rowed subtableaux consisting of rows i,j of T is 
less than the corresponding column word determined by T' iff ct < ct' ■ 

The proof of Algorithm straighten-tableau thus depends solely on the 
correctness of Algorithm row-straighten. We will prove both claimed prop- 
erties of Algorithm row-straighten for each of the two cases appearing in the 
algorithm. First we will produce the "determinantal" identities that will be 
used in Algorithm row-straighten. 

Define a shujfle of a word w = wi,...,Wn into parts of length k, k' — n — k to 
be an ordered pair of words w/ and u/!_ of w having lengths k and k' respectively, 
such that u/ and can be found as a pair of disjoint subwords of w. Neither 
w' nor v/^ need be contiguous as a subword of w. When w — 1, . . . ,n a, shuffle 
amounts to a permutation a of the index set 1, . . . , rt such that ci < • • • < (7^ 
and CTfc+i < • • • < cTfe+fe' . Generalizing the length of a permutation we define 
the shujfle signature, sign{uii, . . . ,ijjk), of a word w to be the number of pairs 
1 !i i < j such that uji > tOj and \uji\ = \ujj \ = I. 

Definition 5.1 Let i,j,k,l be nonnegative integers. Fix a two-rowed row- 
convex shape D by specifying the starting and ending columns, 1 through 
and m through m -\- 1 -\- k — 1 of the top and bottom rows respectively. For con- 
venience we have let the leftmost column index of the top row be 1, but the 
column indices can of course be shifted left or right by any integer. With the 
above convention, we could have m < 0, this produces a skew shape. 

Let T — be a two-rowed row-convex tableau of shape D. Fix two 

sequences of column indices, ci < ■ ■ ■ < Cj starting at column i -\- I -'r I and 
ending at i -\- I -\- j and m < c'l < ■ ■ ■ < c'l < k -\- 1 starting at column ci and 
ending at ci -f Z — 1 . 

Define Syz^.^^ (.j-, c\,....c[{T) to be the formal linear combination 

K+ E ^-^^ (2) 

all shuffles 
x^{i)---x^^iy, 2;^(i+i)...2;^(i+z) 

of VIV2---VC-, -1 



E 



all nontrivial shufHes 

ycr(l) ■■■S/cr(i) ; S/cr(i + l) ■■■y<T(i+j) 
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where T'^, T", and (3r are dehned as follows: 
First, define words 

X^Vi,...,Vci-l, Ui,...,Uj+i = Wc'^, . . . ,Wcl,Vc-,, ■ ■ ■ ,Vcj , 
^ = f^r (1) , ■ • • , Vr(i) , = '"a(l) , ■ • • , "^(O , 

= VT(t+l) , • ■ ■ , Vr(i+l) , = U^{1 + 1) , • • • , Ucr(i+j) , 

£l = . . . , Wc;-1 £2 = Wc; + 1, • ■ • , 

and z — zi,Z2- 

Define the tableau to be the tableau obtained by sorting the rows of 



■■■■X 



and T" to be to be the result of sorting the rows of 



■ ■ u- 
J' 



We define 



c{x)\-c {v/^ ! c (zi, £2) ! • c {v/^ 



with Ni{t) = |£i| • Kl + \z\ ■ {t+j + l) + K\ + + sign(<X) + 

m{x,u'^) + m(£i, mJ^, £2) where m{uj) = sign(cj) + ^^^^^^^ ^i^^ g^id we 

define 



c(u,x']l c(x''z] 



c(u)!-c(^x;j! c(^a^j!-c(i)! 

witi A^2(t) = \z\ + + |<| • ((i + j + - |w|) + sign(<, + m{u,x^) + 
m(x", z). 

Cafi the cells in columns ci, . . . ,Cj of the top row and the cells in columns 
c[,. . . ,c'i of the bottom row of T marked cells. If no positive letter appears 
in both marked and unmarked cells in the top row of T and no positive letter 
appears in both marked and unmarked cells in the bottom row of T then ag = 
±1 so tfie identity holds over Z. 



Proposition 5.2 Let T be a tableau of two-rowed, row-convex shape D whose 
top row contains its bottom row Without loss of generality assume the top 



row has leftmost column index 1 . 
chosen as in Definition 5.1, then 



If column indices ci , . 



and ( 



, C; are 



s 



(3) 
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where J^s'^s ' ^ ~ ^y^ci,...,cj; c[ c[{T) and both sums are over all (w.l.o.g. 

row-standard) tableau of shape D. 

The identity^ also holds when D is a two-rowed skew shape (i.e. AI < in 
Definition 5.1) and I + j exceeds the number of columns in D. In this case, the 
right-hand summand in expression ^ vanishes. 



This result follows from the more general Theorem 10 of [ GRS87 but for 
completeness, we sketch a proof relying on positive letters and the pol arizatio n 
operators of Section ^ that bypasses the Hopf algebra techniques of |GRS87|. 
The proof provides a much simpler though less explicit definition of the expres- 
sions Syzc^c'- 

Proof. First, we prove the following proposition directly by checking that the 
monomials arising from the expansion of each expression have the same coeffi- 
cients. Details may be found in |T97a|. 



Proposition 5.3 Let a, 6, c be positive letters. Let k, I be nonnegative in- 
tegers. Let 1,2, . . . ,i j + I be negative letters. Fix a two-rowed row-convex 
shape D whose top row contains its bottom row by specifying the starting and 
ending columns, 1 through j and m through m -\- 1 -\- k — 1 of the top and 
bottom rows respectively. The following identity holds for tableaux of shape D: 



5(0 c^k) 



12 

m m+l . . . m+l+k-1 

biJ+i) a(0 
ad) c^k) 



where ^...g^... 



Si--- I •••Ti---^ _ 

T 



(-1)' 

::;;) - ( 



1 2 ... 

m m+l 



. . . i+j+l 

m+l+k-l 



Choosing v_ = vi, . . . ,Vi+i, w 
applying the product. 



Wl 



Ti...)(---S2---| 

, . . . , Wfe, and w 



(4) 

T2 • • •) and where 



Ml, . . .,Uj+l. 



and 



1 



c {v)\c (w)!c (u)! 



of polariza tion s to the identity in Proposition 5^ completes the proof when m in 
Definition 5T is positive. For m < 0, we recover the identity used traditionally 
to straighten skew tableaux. A proof follows by recognizing that the right-hand 
side of equation ^ vanishes when j-\-l>i-\-j-\-l — m-\-l. □ 



We have just verified that any formal linear combination of tableau with 
integer coefficients produced by Algorithm row-straighten satisfies Claim 1 
made in the algorithm specifications; we now go to work on the heart of the 
proof, namely Claim 2. In the process of proving Claim 2, we will check that 
row-straighten terminates. 
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nil 



ci 



1 ci 



C2 — 1 C2 A2 



Ai 



m2 C3 
Figure 4: Case I: ci < C2: Relations between entries in a two-row tableau 
being straightened by Algorithm row-straighten. 

All entries in the bottom row from C2 through C3 are equal but distinct from 
any entry in column C3 + 1 of that row. 



Proposition 5.4 Let D be the row-convex shape {Xi, /{mi, 1122) with mi < 

7712. 

GiVeii a non-straight, row standard, two rowed, row-convex tableau, T , of 
shape D Algorithm row-straighten produces a formal linear combination of 
tableaux each of which has a lexicographically larger column word than ct ■ 

Proof. The proof is by induction on ct- 
Suppose that T is the tableau 

Column # mi mi+1 m2 7712+1 A2 Ai 

I'm! Vmi + 1 ■ ■ ■ V\i 

Wm2 Wyn2+l ■■■ WX2 

We set up a straightening syzygy that expresses T in terms of tableau Ti 
such that Ct is always lexically smaller than ct^ . We first describe the structure 
of T with respect to its leftmost flippable inversion. Let q be minimal such 
that if column q + I were to contain an inversion, then that inversion would be 
flippable. Thus q — minm2<i<A2 « — 1- The presence of a flippable inversion 

guarantees that q exists. The value of ci in Algorithm row-straighten is q+l. 
The first column strictly right of ci — 1 that actually has an inversion has index 

C2 = minci<j<A2 j. This inversion must be flippable, thus C2 indexes the column 

„,<+^^, 

of the leftmost flippable inversion in the tableau. 

Two cases arise in the algorithm namely ci < C2 and ci — C2. The pictures 
in Figures |4| and ^ outline these situations. The symbol, "•" , indicates a cell in 
the diagram. An arrow from one cell to another indicates that the contents of 
the first cell are larger than the contents of the second. The decoration of an 
arrow by "— " (respectively "+") indicates that the contents of the cells at either 
end are may be equal if these contents are negatively (respectively positively) 
signed. Sequences of cells surrounded by parentheses or braces may be omitted. 

If no positive letter appears multiple times in T, then Cases I and II can be 
treated simultaneously. We begin with Case I. 



A straighteng algorithm for row-convex tableaux. 



18 



Apply Corollary 5.2 to write 



"'"^ ■■■ ""^^ = Syz =B + A 

Wm2 ■■■ Wc^ ■■■ Wc2 ■■■ ... J c2,...,A2;ci...C3 

(5) 

where B (respectively A) is the first (respectively second) summation in the 
SyZc2,...,\2;ci...c3{T) as defined in expression ||. The over/underlines are visual 
aids which indicate the "marked" entries used to define Syz{T). 

It suffices to show that each tableau appearing in ^ or B has lexically larger 
column word than ct- 

Suppose T' appears in A. We can write 





. . . Xt-l Wci ... Wc3 


VC2 








Wm2 ■■■ 2/1 


2/c2-t 


WC3 + 1 ■ ■ 


■ WX2 



where Xmi ■ ■ ■ xt-i; yi ■ ■ ■ 2/c2-t is a shuffle of Vmi ■ ■ - Vr, where t = C2 + C1—C3 — I, 
and where the boxed entries must be sorted in order to give a row-standard 
tableau. Denote the entries in the bottom row by , . . . , zx^ and the entries 
in the top row by Wmi , • ■ • , ■ 

To check that only the boxed entries need to be sorted in the to row, it suffices 



to observe that the Xi are taken from Vj^-^ , . . . ,Vr 



and that +> Wc2 = Wc 



Checking the bottom row, it suffices to note that Wc3+i +> i'c2-i- 

Now let fc + 1 index the leftmost column in the top row in which T' differs 
from T. In fact, k — minmi<i<t i — 1 which follows from being in Case 1: 

Suppose Wi = Vi for mi <i<t = ci — 1 — C3 + C2. Since by construction 
Vt-i <+ Vci-i < Wen have that Xi — for all i as above and thus the boxed 
elements in the top row are already in order. But then mt ~ ^ ft, since by 
Case I Vci-i < Wci ■ So fc < i. 

Now we examine the column words. Our construction shows w_ > v, so by 
the preceding paragraph w> v. So if fc + 1 < 7712 we conclude directly that ct' 
is lexically larger than ct- 

Suppose that fc + 1 > m2. We show that Vk+i < yi- Suppose to the contrary 
that Vk+i > yi- Since yi comes from Vm^ , . . . ,Vr this says that yi = Vj for some 
j < k and j/i ^ Vji for j' > fc. Now the upper row of T' still contains vi . . .Vk 
even though a yi has been removed to the bottom row. But this implies that yi 
also appears in Wc^ . . . Wc2 which is impossible since Wc^ +> fci-i +> Vk+i > yi- 

Thus since fc + 1 < i < Ci the diagram for Case I shows that Wk+i < 
Vk+ij hence w^+i < yi. So, after sorting, we find that = Wmai -^"12+1 = 
Wm2+i, . . . , Zfc+i — Wk+i- So in tableaux T and T', the columns mi, . . . , fc agree 
as does the bottom entry of column fc + 1. But the top entry in column fc + 1 
is larger in T' than in T. Hence ct' is lexically larger than ct- 

At last we deal with tableaux appearing in B in equation ^ Recall that 
tableaux in B arise from nontrivially shuffling the over/underlined entries and 
then resorting the rows. Let u = Wci • ■ • Wcr^Vc2 • • • . Let y!' ,11!, be a shuffle of 
u into two parts of size Ai — C2-I-I and c^ — ci + l respectively. Since Wc^ +> fci-i, 
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such a tableau will look like 



r 



Wr, 



where, as before, the boxed elements must be sorted so that T' will be row 
standard. Again denote the top and bottom rows of T' by Wmx , • • ■ , and 

-2^7712 ; ■ ■ ■ 1 ^A2 ■ 

Now let A: + 1 be the leftmost column in which the bottom rows of T and T' 
disagree. We claim ci < fc + 1 < C3 and that Zk+i > Wk+i- By construction 



Because 



Write u' = u' 



Let j be minimal such that u' 7^ 



+> Wc2 



this implies that u' > Wj. But since also i^cs+i > 



we find Zj > Wj. and = Wi for all i < j, so fc + 1 = j. 



Subcase 1. Suppose fc < C2 — 1. Any letter appearing in the multiset difference 
u" ~ {{vc2i ■ ■ ■ 1 v\-^ }} is (+>)-greater than Wk+i- But since fc < C2 — 1, the picture 
of case I shows that Vk+i Wk+i, this means that on resorting the boxed 
elements of the top row, every element in u' stays in column fc + 2 or higher. 
Hence columns mi . . . fc agree in T and T' . But the bottom of column fc' + 1 is 
larger in T' than T. Thus ct' is lexically larger than ct- 

The above argument also generates the fact (unused in this proof, but see 
the comment after Corollary 5.7) that, T and T' agree in the top element of 
column fc + 1. 

Subcase 2: suppose that fc > C2 — 1. This says that the bottom rows of T, T' 
agree at least through column C2 — 1 . Since - 1 <+ Wc2 j '^e have immediately 
that the top rows of T, T' agree through column C2 — 1. Now either the bottom 
of column C2 changes (hence increases) so ct' is lexically larger than ct and 
we are done or the the number of positive letters in the bottom row that equal 
decreases. In the latter case, not only do the tableaux T, T' agree up to 
column C2 — 1 but T' still has a flippable inversion in column C2 since the entry 
in the top of that column now equals the positive letter Wc2 that remains at the 
bottom. 

We repeat the straightening law on T', producing some tableaux with lexico- 
graphically larger modified column words and some tableaux that are unchanged 
in columns smaller than C2 and unchanged at the bottom of column C2 but which 
have fewer copies of Wc2 in their bottom rows. Eventually, we must run out of 
positive letters equal to ■Wc2 in the bottom row and so eventually the modified 
column word increases. 



We now treat Case II. Here ci = C2. 
We replace [T] with Syzcp Ai;ci. 



where cq is minimal such that Vcg +> 



Wc 



As before, Corollary 5.2 lets us write 



Wc 



= B + A 



(6) 



just as in equation g The entries that have been marked twice are positive letters 
all equal to each other. 
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mi ^ C2 - 1 C2 A2 Ai 




• ... • 



TO2 C3 

Figure 5: Case II: ci =02'. Relations between entries in a two-row tableau 
being straightened by Algorithm row-straighten. 

All entries in the bottom row from C2 through C3 are equal but distinct from 
any entry in column C3 + 1 of that row. If cq < C2, then the entries in the top 
row that equal the bottom row entry in column C2 must start at cq and extend 
at least as far as C2 — 1. 



Suppose T' appears in A. To this purpose, let w = . . .v^ be the word 
being shuffled. It is easily verified that 

-^mi ■ ■ ■ — 1 ^co — ■ ■ ■ — ■ • • ■ ■ • ^Ai 



Wm2 ■■■ ■■■ Wc2-1 Vl ■■■ Vco-t 



Wc3+1 ... WA2 



where t = cq - C3 + C2 - 1, ■ ■ ■ Xt-i; yi ■ ■ ■ Vco-t is a shuffle of Vmi , ■ ■ • , Vco-i 
and where the boxed entries must be sorted in order to get a row-standard 
tableau. Maintain the notation T' ~ 

■■■£■■■ 

Since i < cq we have 7^ vt and thus fc = min,ni<i<t i — 1 is well defined. 
As in Case I, if fc + 1 < m^ we conclude directly that ct' is lexically larger than 

CT- 

Suppose that fc + 1 > TO2. We show that ffc+i < y\. Suppose to the contrary 
that Ufc+i > y\. Since y\ comes from fmi, ■ ■ • ,i'co-i this says that y\ = Vj for 
some j < k and yi 7^ Vj/ for j' > fc. But this says that if the letter yi occurs 
in the yi ■ ■ ■ yca-t part of the shuffle, then the a;,„j ■ ■ ■ Xk part cannot start with 
• ■ ■ Wfc-contradiction. 

Thus since the diagram for Case II shows that Wk+i < I'fc+i, we find Wk+i < 
2/1. So, after sorting, we discover that 

Wk+i- So in tableaux T, T', the columns toi, . . . , fc agree as does the bottom 
entry of column fc + 1 . But the top entry in column fc -|- 1 is larger in T' than 
in T. Hence ct' is lexically larger than ct- 

Suppose now that T' appears in B in equation^. Define C4 = minc2<i<Ai i. 

™c2 

Since +> i'c2-i = ''^coi we have 

Column: C4+S-1 

T' = '"mi •■• Wc2-1 Wc2 ^ •■• ^Wc2 W 

Wm2 •■• Wc2-1 Wc2+s ■■■ W. 



where VF', W" is a shuffle of Vc^ - ■ ■ into parts of size Ai — C4 — s + 1 and 
l<s<C3 — C2 + I respectively and, as before, the boxed elements must be 
sorted so that T' will be row standard. 
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The top rows of T, T' agree through column C2 — 1. Again either the bottom 
of column C2 increases and we are done or the the number of positive letters in 
the bottom row that equal decreases. So iterating the straightening law on 
T' eventually increases the modified column word. 

□ 

The algorithm row-straighten specializes to the classical straightening for 
skew and partition shaped tableau when nii > TO2- The preceding result implies 
that the column word also increases in the skew case. 



Corollary 5.5 Proposition 5.4 also holds with mi > ni2 



Proof. We use what is often called the method of fake letters. Fill cells 
TO2 — 1, ... , mi — 1 in the the top row with new negative letters disjoint from 
and smaller than the letters in £, we will name these letters /m^-i, . . . , fmi-i- 
Straighten this new tableau. 

The tableaux appearing in expression B in the preceding proof have the 
"fake" letters /m^ _ i , . . . , /,„^ _ i in the same positions as does the original tableau 
T. If we apply the algebra homomorphism that sending to 5ij, \T'] is sent 

to for all T' in the expression A in the preceding proof and the fake letters 
are erased from all other tableaux in the expression. □ 

We have now established the correctness of Algorithm row-straighten. 

Theorem 5.6 The straight tableaux of shape D form a Z-basis for and 
Algorithm straighten-tableau expands any generator of in terms of this 
basis. Further, given a row-standard tableau T, the expansion of [T] is in terms 
of tableaux with larger column words than wt. D 



By Corollary 4.15 carefully analyzing the proof Proposition 5.4 we can extend 



the preceding result. 

Corollary 5.7 AJgoritiims row-straightening and straighten-tableau pro- 
duce tableaux with weakly larger modified column words than that of the input 
tableau. □ 



6 Flagged super-Schur modules 



The flagged S chur modules Sf ha ve been the subject of considerable interest (see 
for instance | LS90 , EIS96 , LM97 |). We apply the preceding results to flagged 
super-Schur modules of row-convex shape. The fact that the initial terms in 
the straight basis are distinct allow the straight bases to descend to bases of the 
corresponding flagged module. I will start by formalizing the notion of a flagged 
superSchur module. 
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Definition 6.1 Let f be a weakly increasing sequence of letters in the alphabet 
£. Regard this sequence as indexed by elements ofV. The flagged superSchur 
module [C) is the subquotient of Super {[£ \ V]) equal to the image of the 

submodule S^{C) under the map 0/ which quotients Super {[C {V]) by setting 
{l\p) = whenever I > fp. 

A tableau T is flagged if, in each column i, T has no entry exceeding fi. 

If T is row-standard (of any shape) and fails to be flagged, then 0/([r]) = 0- 
since each monomial in the expansion of any row in which the flagging condition 
is violated has some fact or {l\p) with I > fp. 

Classical results (see |Sta76 |) tell us that if is a skew-tableau then a basis 



for Sj^ is given by all [T] such that T is standard and flagged. This result carries 
over to flagged row-convex superSchur modules. 

Theorem 6.2 Let D be a row-convex shape. Fix a weakly increasing flag f . A 
basis for S^ {£) is given by the elements 4'f{[T]) where T runs over all flagged, 
straight tableaux of shape D with entries chosen from C 

Proof. It suffices to show that the basis elements are linearly independent, 
indeed that their initial ter ms un der any diagonal term order are still distinct. 



This follows by Proposition [4.12| , and the observation immediately following it 



since the tableaux T are both straight and flagged. □ 



This result has the following easy generalization. Let f,g both be weakly 
increasing sequences of letters in C indexed by elements of V such that f < g 
componentwise. Define the doubly flagged superSchur module Sf g[C) to be 
the image of S^{C) under the map quotienting Super{[C \ V]) by the ideal 
generated by {(Z|p) :/ ^ /;,... , gi}. Call a tableau T doubly flagged with respect 
to /, g if every entry in column i is between fi and gi. The same proof as above 
shows the following. 

Theorem 6.3 A basis for Sj^g{C) is given by the elements (j)f{[T]) where T 
runs over all doubly flagged, straight tableaux of shape D with entries chosen 
from C. □ 



When C = C and D is skew, this results appears in |Sta76 



7 Groebner and SAGBI bases. 

The basis theorems developed above have various ring-theoretic applications. 
For convenience, we state them in terms of commutative rings, i.e. we assume 
that the alphabet C consists entirely of negative letters. We can then take 
the subalgebra of the polynomial algebra k[a:;.ij] generated by all polynomials 
indexed by tableaux of some fixed shape. 
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Definition 7.1 Let D be a shape. Let be the subalgebra generated by all 
[T] for all tableaux T of shape D. Allowing that [T] has degree 1, this is a 
graded algebra generated by its degree 1 part. 

These algebras turn out to be the homogeneous coordinate rings of certain 



configuration varieties (see [M98 for example) embedded in projective space 
by a combination of Plucker embeddings, Segre products and Veronese maps. 
Configuration varieties parameterize a tuple of subspaces of n-space subject 
to certain lower bounds on the dimensions in which they may intersect. The 
results in this section are stated for the rings but they hold equally for the 
subquotient rings generated by the (doubly) flagged modules ^{C). 

Below we produce a Groebner basis for R^ . First we establish some notation. 

Definition 7.2 Let D be a shape. Let T,T' be tableaux of shape D. Dehne 
T oT' to be the tableau (no longer of shape D ) formed by alternating rows of 
T and T' , starting with the top row ofT, then the top row ofT' etc. 

Before stating the theorem we recall our convention that the initial term of a 
polynomial is the smallest term in the polynomial and define the degree of a 
Groebner basis be the highest degree of a polynomial appearing in Groebner 
basis. 

Theorem 7.3 Let D be an n-rowed, row-convex shape. 

Consider the polynomial ring whose variables consist of all straight tableaux 
of shape D on C. We define a graded term order on this ring as follows. We 
say T' < T" when ct < ct" and we define HLi ^/ < Hj^i ^i' where r < s 
implies r; < T; and T/' < T," when c^{T[), . . . , c^{T[) , c^{T'^) , . . . ,c^{T[), . . . 
is smaller in lexicographic order than c-^{T'j!), . . . , Cj(T"), 02(1"^'); ■ • ■ ; ^2(1""), . . . 
where c^(T) is the ith column ofT read from bottom to top. 

If I is the kernel of the ring map sending a straight tableau T to [T] , then I 
has a degree 2 Groebner basis consisting of all polynomials 



T'-T" - 5^/3.. . '"^'t'"' (7) 

^ ... ^ .^^ ... . . . . . . 



where T', T" range over all straight tableaux of shape D on C and where 

EPk ■ — = SyZaj^,„ a^;bi b 

■ ■ ■ m.. ■ ■ ■ . , , . . 
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if "rE* " is not skew a and b are allowed to range over all sequences of consecutive 
column indices such that a ends in the last column of t[ and if ...rj^'..' is skew, 
a and b range over all sequences of column indices such that r + s is large than 

the number of columns in 'rf!' " . 

■ ■ ■ 1 j ■ ■ ■ 

Proof. Let T < T' be straight tableaux of shape D. Corollary verified 
that the polynomials ^ are in /. It suffices to show that if T := T' o T" is not 
straight then there exists a relation in the Groebner basis |^ whose initial term 
is T' o T". Since T is not straight, there exists i < j such that if is the ith 
row of T and t" is the jth row of T' then the tableau is not straight. 



But then Proposition 5.4 and Corollary 5.5 show that choosing a and b as in 
algorithm row- straight en gives T' ■ T" as the initial term of the polynomial ^. 

□ 



The Groebner basis of Theorem |7.3| is not reduced, nor are the initial terms 
of its elements polynomials necessarily distinct. The initial terms can be made 
distinct by choosing a and b as in algorithm row-straighten. The above the- 
orem does not require that C =^ Cr . although one requires the notion of a 
non-commutative Groebner basis for general L. Specifically when L = Ct , it is 
possible to restrict r -I- s to be one more than the maximum of the number of 
columns in and the number of columns in Tj while simultaneously eliminating 
any other restrictions on the indices a and 6; the underlying straightening law 



is presented in Chapter III of | T97a |. In general, that straightening law fails £ 
contains positive letters. 

Porism 7.4 The monomial Ti-T2 . . .Tk is standard with respect to the Groeb- 



ner basis in Theorem l.c, iff Ti o T2 o . . . o T^ is straight. □ 



The existence of a degree 2 Groebner basis for an algebra is known to imply 
that there is an (infinite) linear free resolution of the ground field over the 

algebra. 

A SAGBI (Subalgebra Analogue of a Groebner Basis for Ideals) basis, see [ KaMa89| | 



and |RoSw90|, is a generating set for a subalgebra such that the initial terms 
of the subalgebra are contained in the algebra generated by the initial terms of 
the generating set. 

Theorem 7.5 Let D be a row-convex shape. The straight basis elements of 
shape D form a SAGBI basis for with respect to any diagonal term order. 

Proof. It suffices to show that if p is in then its initial term is the product 
of the initial terms of some multiset of straight tableaux. Define D°'' to be the 
shape formed by replacing each row of D with k copies of itself. So init (p) is 
init (pk) where pk is the component ofp lying in with k maximal such that 
Pk ^ 0. Since the initial terms of straight tableaux of fixed shape are distinct. 
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init (p) — [T] for some straight tableaux T of shape D° . But we can write 
T = Ti o • • • o Tfc. Each T, must be straight and init ([T]) = JlLi i^it {[Ti]) □ 



Corollary 7.6 Let D be a row-convex shape. The row-standard tableaux of 
shape D form a SAGBI basis for with respect to any diagonal term order. □ 

By the usual results these SAGBI bases give algorithms for determining 
whether a polynomial in variables Xij belongs to R^{C) or (a forteriori) S^{C) 
and, if so, writing it in terms of the generators [T] where T is straight of shape 
D. By results of |Stu9€], a SAGBI basis for an algebra allows that algebra to 
be deformed to an algebra generated by monomials. In |T97a] and [r99| this 
deformation is used to prove that the subalgebra generated by all tableaux of a 
fixed row-convex shape is Cohen-Macaulay. 



8 A branching rule and flagged corner-cell re- 
currence. 

Our final application concerns a branching rule for row-convex representations. 
The Schur and Weyl modules S^{C^) and S^{C^) are GL„ representations 
with Gin-action induced by the algebra homomorphism g : Super ([C | "P]) — *■ 
Super{[C I P]) given by (7((r|s)) — where g £ Gi„ equals (gij)- In 

order to handle sets C containing letters of both positive and negative sign, we 
will work with representations of the general linear Lie superalgebra, pl^. 

We express a pZ£-representation, corresponding to a row-convex shape Z?, in 
terms of plc\{a} representations (for some a £ C) corresponding to subshapes of 
D. The combinatorics for the case C = is identical to that of the branching 
rule in |RS98| and new when C = C~ . We present a filtration that realizes 
this branching rule in a characteristic-free fashion. This provides the row- 



convex case of filtration conjectured in [RS98| to exist for all % comment deltd 
It should be noted that the orientations of [RS98| are at variance from those 
of [EIS95 ; we adhere to the orientation of the latter. Thus the term row-convex 



in [RS98 should be read as "column-convex" in the context of both [RS95| and 
the present paper. The branching rule presented below generalizes to the case 
of flagged super-Schur modules; branching rules for flagged Schur modules are 
not treated in plS98[. 



First we construct the general linear Lie superalgebras following Scheunert |Sc79| 
A free Z-module F is signed when it has distinguished free submodules Fq 

and Fi whose direct sum is F. Elements of Fo and Fi are called homogeneous 

and |a;| = j for x € Fi. 

A free signed Z-module is a Lie superalgebra when it is endowed with a 

superbracket [ , ] satisfying the commutativity relation. 
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for homogeneous elements x, y and the super- Jacobi identity 

(-l)l''ll^l[a, [6, c]] + (-1)I°H''I[6, [c, a]] + (-l)l''ll=l[c, [a, b]] = 
for homogeneous elements a, 6, c. 



Following | BT91 , the general linear Lie superalgebra pZ^, associated to the 
signed alphabet, C, is the vector space (over Q) with basis i?a,b for a, 6 G C, 
where |i?a,b| = \a\ + \h\ and the bracket is 

We next describe an action of i?a,b on Super{[C \ V]). 

A (left) superderivation D on a superalgebra j4 is a Z-linear endomorphism of 
A such that for p, q homogeneous in the Z2 grading of A, the identity D{pq) = 
{Dp)q + {—iy^P^p{Dq) holds for some fixed e G Z2. This e is the sign of D, 
written \D\. 

We define the letter polarization Da,b ■ Superci([C | "P]) ^ SuperQ([C \ V]) 
to be the superderivation with sign \a\ + \b\ such that Da^b{c\p) — ^b,c{o-\p) 
where S is the Kronecker delta. It is easy to check that these superderivations 
are well-defined on the Z-subalgebra Super {[C \V]). 

The next example describes the action of the polarization operators in the 
case that the biproduct is the determinant of a minor. 

Example 8.1 If £ = and V — , then Super{[C \ V]) is isomorphic to 
Z[a;ij : i ^ C, j <E P]. The action of Dij (respectively ijR on this algebra is 
given by T,pev^^.Pdi- (respectively J^iec'^L^di^)- 

To make our results characteristic free, we work over U {pic): the Z-subalgebra 
of the universal enveloping superalgebra of pl^ generated by all -Ea.b, by 
for all i e N and all a 7^ 6 such that |a| = |6|, and by all (^'J "), for i G N. 

Proposition 8.1 The map Eafi i~> Da,b, provides a representation ofU{plL) 
on Super {[C {V]). 

If D is a shape such that any letter appearing in Der~ (D) appears in V , 
then this action descends to an action on S^ (C) . 

Proof Sketch. 

Defining the polarization operators to be superderivations on Super{C), it 
can be shown that the polarization operators satisfy Da,b{p\q) = {Da,bP\Q) 
where ( | ) is the bilinear form of Definition ^.2| . It is then clear that S^ (C) is 
closed under the action of the superderivations. 



Details may be found in |T97a|. □ 



The branching rule for S^ involves removing vertical or horizontal strips 
from D. 

Definition 8.2 Let D be a sorted row-convex shape. Define a horizontal strip, 
in D to be any subset of the cells of D such that there exists a shape D 
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straight tableau, T, on some alphabet a"*" < 61 < 62 < • • • where the cells in T 
that contain a"*" are precisely the cells of E. Similarly, define a vertical strip, 
E~ as any set of cells containing all the negative letters a~ appearing in some 
straight tableau of shape D on some alphabet a~ < 61 < 62 < • • •. 

Let g, f be two weakly increasing sequences of letters indexed by the elements 
ofV. A vertical or horizontal strip is a-flagged (with respect tog, f) if it contains 
cells only in columns i where gt < a < fi. 

Note that strips are allowed to be empty. 

Lemma 8.3 Let E be a vertical (respectively horizontal) strip in a row-convex 

shape D. Let Ie be the multiset of column indices indicating in which columns 
the cells of the strip appear. If E' is another vertical (respectively horizontal) 
strip, then Te = 1e' implies E = E' . 

Proof. We utilize an alternative to Algorithm Straight-Filling for producing 
straight tableau with specified column content and shape. Suppose D has n 
cells. Define the desired contents of the columns of a tableau by a biword 

u = (^[[[■^,,,^ where u = W£,er-{D) Ui <- ttj+i if iii = Define a 

biword (^[[[■^,,,^ = (^^"JJ^j' by permuting the entries of u so that w weakly 

increases and so Wi = Wj+i implies Wi < Wj+i; when [wil = 0, this inequality is 
strict. 

Using this biword w, wc fill the tableau by starting with an empty tableau of 
shape D and adding successive letters reading left to right through the biword. 
At step j we place wj in the northmost available cell (say row i) in column Wj 
such that cither (i,Wj — 1) is not in the diagram or such that the cell {i,Wj — 1) 
contains a letter x with x <+ Wj. If no such cell exists, then the biword does 
not arise from a straight tableau of the given shape. To verify this algorithm, 
observe that if wc put Wj into another row i' , then cither row-standardness is 
violated or we have created a flippable inversion in cells {i,Wj) and {i',Wj). 

The lemma is an immediate consequence of the algorithm's correctness. □ 



Definition 8.4 Suppose that D is a row-convex shape, £ is an alphabet, and 
Z[ti : I £ £] is a polynomial ring. Let g, f be two weakly increasing sequences 
in C indexed by the elements of V. Define 

</(^)= E n ^^(..) 

T straight {i.j)^D 

where the sum runs over all g, f -doubly flagged straight tableaux of shape D on 
C and where T(^ij) is the {i,j)th entry ofT. 

When / and g are trivial, that is they contain respectively only the largest and 
smallest elements of £, and when £ contains letters of only one sign, this is 
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the formal character of the Gid^D-representation S^{C). If just one of f,g is 
trivial, we get the formal character of a representation of a Borel subgroup. 

The following identity is immediate from the definition of a straight tableau, 
when C = C+ it is due to |RS9^ . 



Proposition 8.5 Fix two weakly increasing sequences g ^ f of letters, and choose 
a G C. If D is a sorted row-convex diagram, then 



</(^) = E<f (AW) 



where the sum runs over all a- flagged horizontal (respectively vertical) strips 
E in D when a € £ is positive (respectively negative), and where D/E is the 
diagram formed by removing E from D . D 

Preparatory to establishing a filtration for p/^-modules (£) that realizes 
this identity we define some components of that filtration. 

Deflnition 8.6 If E, E' are two vertical (or two horizontal) strips in D, define 
E < E' in dominance order when for all i, the number (counted with multiplic- 
ity) of elements in {1, ...,?} in Is is at least as large as the number of times 
these elements appear in Xe' ■ 

Let E be a vertical (respectively horizontal) strip and let a € C be negatively 
(respectively positively) signed. Dehne 

a)=spanz{[T]} 

T 

where T runs over all shape D tableaux on C in which a appears in a vertical 
strip E' weakly dominating E. 

Define S^D, > E{C; a) identically except for the requirement that E' must 
strictly dominate E. 

It is immediate from the definition that S^D, > E{C; a) and S^D, > E{C; a) 
are p/^^^^^j^-representations. 

Theorem 8.7 Let D be a row-convex shape. If a is a negatively (respectively 
positively) signed letter in C and E is a horizontal (respectively vertical) strip 
in D, then 

S'^'^^iC; a) I 5^^>^(/:; a) ^ 5^/^(AW) 

as a pi ^\^^g^y representation. Here D/E is the shape formed by removing E from 
D. 

Proof. It suffices to observe that given a row-standard tableau T such that the 
cells occupied by a comprise E, then any tableaux appearing in the straightened 
form of [T] has the cells occupied by a form a strip E' determined by a multiset 
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I' > I. This can be seen by directly examining the straightening relations. 
In particular, any straightening relation which moves the a's produces a row- 
standard tableau in which the a's form a horizontal (respectively vertical) strip 
indexed by some I' > I. □ 

A more sophisticated result on the allowable contents of a tableau appearing 



in the straightening of [T] is proved in |T97a| Chapter III, Section 6 



Corollary 8.8 Let D be a row-convex shape, and let a be a negatively (respec- 
tively positively) signed letter in C. Let Ei, . . . ,Ek be all vertical (respectively 
horizontal) strips in D ordered compatibly with dominance, so that i > j implies 
Ej ^ Ek- The Rltration 

k k 

^"'^^ (-C; a) D • • • D ^ S^'^^ (/:; a) D • • • D S^'^" {C; a) 3 

has S^/^^{C\{a\) as the quotient, up to pi ^\^^^y isomorphism, of its jth term 
by its j + 1st term. □ 

If C is sufficiently large, then the containments in the above filtration are all 
strict. 

The preceding results generalize immediately to the iSn-representations pro- 
vided by the Specht modules. 

If we define S^'^'{£; a) to be the flagged super-Schur module found by 

taking the image of S^'^'(C; a) under the map {l\p) •— > when I > fp, then we 
have the following. 



Proposition 8.9 Maintaining the notation of Corollary S.t, the filtration 

k k 

5f a) ^ ■ ■ ■ 2^Sf'^' (C; a) ^ ■ ■ ■ ^ Sf'^" (C; a) 3 

i=l i=j 

of Sj' by B-modules has S^^^' {C\{a]) as the quotient, up to isomorphism, of 
its jth term by its j -\- 1st term. Here B is the subalgebra ofU(plL) generated 
by all El^Jil for b > a and all ('^°'") . □ 

When C = , these results generalize to quantum Schur modules, details 
appear in lT97a[ | and |T99| . 
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